Trace Refinement in Labelled Markov Decision Processes
نویسندگان
چکیده
Given two labelled Markov decision processes (MDPs), the trace-refinement problem asks whether for all strategies of the first MDP there exists a strategy of the second MDP such that the induced labelled Markov chains are trace-equivalent. We show that this problem is decidable in polynomial time if the second MDP is a Markov chain. The algorithm is based on new results on a particular notion of bisimulation between distributions over the states. However, we show that the general trace-refinement problem is undecidable, even if the first MDP is a Markov chain. Decidability of those problems has been open since 2008. We further study the decidability and complexity of the trace-refinement problem provided that the strategies are restricted to be memoryless.
منابع مشابه
Trace Equivalence Characterization Through Reinforcement Learning
In the context of probabilistic verification, we provide a new notion of trace-equivalence divergence between pairs of Labelled Markov processes. This divergence corresponds to the optimal value of a particular derived Markov Decision Process. It can therefore be estimated by Reinforcement Learning methods. Moreover, we provide some PACguarantees on this estimation.
متن کاملAccelerated decomposition techniques for large discounted Markov decision processes
Many hierarchical techniques to solve large Markov decision processes (MDPs) are based on the partition of the state space into strongly connected components (SCCs) that can be classified into some levels. In each level, smaller problems named restricted MDPs are solved, and then these partial solutions are combined to obtain the global solution. In this paper, we first propose a novel algorith...
متن کاملAbstraction and Refinement for Solving Continuous Markov Decision Processes
ion and Refinement for Solving Continuous Markov Decision Processes Alberto Reyesand Pablo Ibargüengoytia Inst. de Inv. Eléctricas Av. Reforma 113, Palmira, Cuernavaca, Mor., México {areyes,pibar}@iie.org,mx L. Enrique Sucar and Eduardo Morales INAOE Luis Enrique Erro 1, Sta. Ma. Tonantzintla, Pue., México {esucar,emorales}@inaoep.mx
متن کاملLabelled Markov Processes as Generalised Stochastic Relations
Labelled Markov processes (LMPs) are labelled transition systems in which each transition has an associated probability. In this paper we present a universal LMP as the spectrum of a commutative C-algebra consisting of formal linear combinations of labelled trees. This yields a simple trace-tree semantics for LMPs that is fully abstract with respect to probabilistic bisimilarity. We also consid...
متن کاملPerformance-Oriented Refinement
We introduce the probabilistic action system formalism which combines refinement with performance. Performance is expressed by means of probability and expected costs. Probability is needed to express uncertainty present in physical environments. Expected costs express physical or abstract quantities that describe a system. They encode the performance objective. The behaviour of probabilistic a...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2016